⚠️ GitHub issue #49905 has been automatically assigned in GitHub to PR creator.

Thanks a lot @AntoinePrv .

@jorisvandenbossche Could you please take a quick look at this? (also is there a way to make Pandas behavior more predictable here?)

Hmm, that's annoying.

I don't directly understand why this is failing, though (but I can reproduce it by setting sys.modules['pyarrow'] = None first and then important pandas locally). In pandas, in cython code, we essentially do this:

cdef bint PYARROW_INSTALLED = False

try:
    import pyarrow as pa

    PYARROW_INSTALLED = True
except ImportError:
    pa = None

Trying to import pyarrow after sys.modules['pyarrow'] = None does raise a ModuleNotFoundError, which is a subclass of ImportError, and so I think should be correctly caught by the block above?
Unless that code does not behave exactly as expected when in cython.

@jorisvandenbossche I can't reproduce this with Pandas 3:

>>> import sys
>>> sys.modules['pyarrow']=None
>>> import pandas
>>> pandas.__version__
'3.0.2'
>>> import pyarrow
Traceback (most recent call last):
  File "<python-input-9>", line 1, in <module>
    import pyarrow
ModuleNotFoundError: import of pyarrow halted; None in sys.modules

@raulcd it does not yet happen on import, but on first usage that checks input for being a pyarrow array, such as pd.DataFrame({"col": [1, 2, 3]})

Essentially, I think this boils down to python imports in cython working a bit differently. In pure python, import pyarrow raises a ModuleNotFoundError if you set the module to None in sys.modules. But in cython, it actually "finds" the module (it doesn't raise an error) but sets variable for the imported module to None (i.e. the value that is set in sys.modules).

We could specifically check for that case on the pandas side, although that feels a bit of a hack. We could also move that try/except to python code, so it works more as expected.

@jorisvandenbossche what about the proposed fix of this PR? It does solve the issue.

Ah, yes, I am certainly fine with that workaround to have archery working (it just should not be necessary ..)

After merging your PR, Conbench analyzed the 0 benchmarking runs that have been run so far on merge-commit 61c96ca.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.